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Self -Critical  amova  with  Application  to  Prothrombin  Time 


Summary 

Tha  purpose  of  this  work  ml*  to  develop  procedures  which  would  improve  tha 
uniformity  of  tha  gradint  procadura  in  tha  hamatology  profldancy  tasting  program  of 
tha  Hew  York  stata  Department  of  Haalth .  Approximately  400  laboratorlaa  In  New  York 
parti  cl  pata  In  a  program  of  clinical  laboratory  profldancy  tasting ,  with  aach 
laboratory  using  ona  of  sight  mathods  and  ona  af  rvlna  thromboplastins .  Tha  rsspormss 
of  aach  tasting  program  may  ba  modalad  as  a  two-way  Layout,  soma  calls  of  which  ara 
ampty.  A  modal -critical  analysis  of  varlanca  tachniqua  was  usad  to  datarmins 

simultaneously,  and  In  tha  prasanca  of  out -of -control  laboratorlaa  l.a. ,  outliers ,  tha 
affects  of  method  and  thromboplastin  on  prothrombin  time.  Tha  word  nodal -critical 

Indicates  that,  according  to  tha  modal ,  eat  up  Initially  on  a  tentative  basis  so  as  to 
allow  for  further  evolution,  tha  responses  should  have  tha  structure  of  a  two-way 
layout  with  Interaction ,  and  that  tha  responses  should  have  a  common  error 

distribution .  This  tentative  framework  is  examined  critically  by  varying  tha  way 
observational  information  la  processed  to  produce  parametric  summaries .  if  the 
summarixations  are  Insensitive  to  the  variation  in  Information  processing ,  then  the 
tentative  model  stands ;  if  not,  the  tentative  model  must  ba  evolved.  we  provide  an 
objective  means  of  statistically  assessing  variations  in  summarization . 

The  model -critical  analysis  produced  a  common  standard  deviation,  identified 
out -of -control  laboratories  and  produced  a  narrower  acceptable  range  of  reported 
prothrombin  times  and  thus  Improved  the  efficiency  of  tha  grading  procadura.  Por 

proficiency  tasting  no  advantage  was  found  in  tha  use  of  either  a  common 
thromboplastin  or  freeze -dried ,  coumarinizad  patient  plasmas  rather  than  artlfically 
depleted  comma rl cal  plasmas,  except  for  special  purposes. 

Keywords i  proficiency  tasting  i  modal -critical  estimation;  outliers,  prothrombin 
time ;  generalized  likelihood,  information  divergence,  thromboplastins 
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1 .  INTRODUCTOOH 


The  N«w  Tort  state  Department  of  Health  operate*  a  prograa  at  clinical 
laboratory  proficiency  testing.  m  this  teeting  prograa,  aaaplee  are  sent  periodically 
in  mail  out  kite  to  all  laboratories  teeting  human  epedaene  froa  the  residents  of  the 
State.  for  many  clinical  saaple  types,  it  is  not  possible  to  prepare  a  saaple  in 
which  the  paraaeters  are  known  in  advance t  they  must  be  estiaated  froa  the  data. 
The  results  of  the  test  are  graded  on  two  bases.  If  any  result  contains  an  error  of 
such  a  magnitude  as  to  Jeopardise  the  well-being  at  a  patient,  the  result  is 
unsatisfactory.  m  order  to  detect  problems  before  they  adversely  affect  a  patient's 
health,  statistical  quality  control  concepts  are  also  used  as  a  basts  for  grading. 
Under  this  combination  of  concepts,  one  would  hope  to  find  that  the  bulk  of  the  results 
would  follow  a  well-behaved  error  model  with  a  small  number  of  outliers  that  reprssent 
the  out -of -control  laboratories. 

In  this  report  we  will  focus  on  the  testing  of  proficiency  of  prothrombin  time,  a 
measure  at  plasma  clotting  time.  Prothrombin  time  is  used  principally  to  monitor  the 
status  of  patients  being  treated  by  oral  anticoagulant  drug  therapy.  An  overdoeage  of 
an  anticoagulant  such  as  warfarin  may  lead  to  hemorrhagic  complications,  while 
underdosage  Increases  the  risk  of  thromboembolic  complications.  This  test  is  also 
used  routinely  to  screen  patients  to  detect  those  at  risk  of  bleeding  excessively.  For 
both  purposes,  the  prothrombin  time  must  be  accurately  standardized  and  correctly 
interpreted. 

The  effect  of  thromboplastin  on  prothrombin  time  has  been  studied  by  a  number  at 
Investigators  (Loellger  et  al.  (1984)}  Biggs  and  Denson  (1967);  Digram  and  Hills 
(1976)1  Loellger  et  al.  (1976)  (  Poller  (1975)).  Reference  thromboplastins  for 
calibration  have  recently  become  available  (Bermans  and  von  den  Bsase laar  (1983)} 


von  den 


>laar  et  al.  (1984))  but  these  have  been  und  Infrequently  in  the  United 


states.  ttie  simultaneous  effects  of  method  and  thromboplastin  have  also  been 
reported  Evatt  et  al.  (1981)  j  Triplett  et  al.  ( 1984)  t  von  den  Beeselaar  et  al. 
(1984)).  Proficiency  testing  is  complicated  by  the  fact  that  prothrombin  time 
measurements  vary  not  only  with  the  skill  of  the  performer  but  also  with  the  method 
and  the  thromboplastin,  a  clot  initiation  reagent,  used,  in  proficiency  testing,  it  is 
necessary  to  take  into  account  structural  effects  of  method  and  thrombol plastln  as  well 
as  the  error  model  while  avoiding  the  excessive  influence  of  the  outlying,  or  out  erf 
control,  observations.  The  model  we  make  use  of  in  order  to  account  for  all  the 
inter-relationships  is  a  two-way  analysis  of  variance  with  interaction  and  a  large 
proportion  of  empty  cells. 

In  the  next  section,  we  will  develop  a  model -critical  or  self -critical  estimation 
procedure .  The  specifics  of  materials  and  data  are  given  in  later  sections .  The 
fourth  section  describes  our  application  of  the  self -critical  method  to  the  prothrombin 
proficiency  testing  problem. 

2.  Model -critical  or  Self -Critical  Analysis  of  variance 


We  provide  first  a  generalisation  of  the  log  likelihood  for  the  normal  distribution 
and  then  generalise  this  log  likelihood  to  a  two-way  layout  with  interaction  and  empty 
cells.  Let  f(xi  n,az)  represent  the  Gaussian  density  with  mean  m  and  variance  a*. 
It  is  easy  to  show  that 


)dx  =  Q(  m , <r*  , c  ) 


C(  2TTCT*  )C(  1+C  )]'*, 


(2.1) 


-1  <  c  <  ®,  is  the  information  generating  function  (Golomb,  1966)  cf  f(x).  For 


rotational  convenience  we  shall  often  delete  the  arguments  of  functions  when 


misinterpretation  Is  not  possible)  for  example,  we  win  sometime*  write  f(x)  or  simply 
f  for  f(st  Observe  that  Q(ti,aM,0)  =  1  and  that  the  Information -theoretic 

properties  of  f(s)  nay  be  developed  from  Q,  for  example  the  entropy  of  f(xt  m.<t* ) 
is  -  ,0)/9 c.  our  main  objective  is  to  develop  a  model -critical  estimation 

procedure  for  4  and  <r*  o €  f(x)  based  on  the  equation  (2.1).  This  is  accomplished 
by  rearranging  (2.1)  to  get 

rfl+C 

x  —  d*  =  l‘  (22> 

If  we  differentiate  (2.2)  with  respect  to  e  *  4  and  a 1  get 

r.  «{*•[«»-*  ^  -  «• 


The  construction  embodied  in  (2.2)  and  (2.3)  shows  that  if  x  is  a  random 
variable  with  density  f,  then  the  expectation 


‘M 


(14C) 


alogf  JlogQ 


(2.4) 


Accordingly,  if  xltxx,...,  xn  is  a  random  sample  from  f(x>  n,o*),  then 
setting  0=4  and  0  =  <j*  in 


,£*<«.,[(«»  >JSS^  ■  “3*]  '  ». 


forms  a  set  of  estimating  equations  for  0  =  4  and  cr2 .  The  line  of  argument  from 
(2.1)  to  (2.5)  is  parallel  to  that  of  maximum  Likelihood  ( see  for  example ,  Kendall 
and  Stuart,  Vol.  U,  pp.  8-10).  Indeed,  the  equations  (2.5)  are  those  of  the  (log) 
likelihood  estimators  when  c=0 .  The  estimators  for  4  and  a1  satisfy  the  implicit 
equations 

n 

M  =  E  w,(c)  xt , 

1=1 

n 

<y*  =  (1+c)  E  w,(c)  (x,-4)2,  (2.6) 

1  =1 

n 

v,(c)  =  fc(x,!  4,<t*),  ▼  (C)  =  £  V,(C). 

1=1 


W,(C)  =  V,(C)/V.(C) 


The  estimator*  for  n  and  <r',  n(c)  and  a*  (c)  say,  havs  been  constructed  so  as  to 


impose  an  adaptive  "Gaussian  screen"  or  "template"  on  the  data.  The  action  of  this 
screen  is  accomplished  through  the  factor  ^(x)  in  (2.3)  and  through  f^x, )  in 
(2.5).  The  screen  Is  adaptive  because  the  values  of  ji  and  c*  are  not  Known  but  in 
the  process  of  Iterating  In  (2.6)  the  estimators  m(c)  and  cr*(c)  finally  settle  on 
values  which  are  most  consistent  with  the  worKlng  model  of  Gausslanlty  and  the  data. 
If,  for  example,  an  outlying  observation  x t  not  consistent  with  Gausslanlty  is  present 
In  the  set  xt,xz,...,  xn,  then  the  screen  will  impose  the  weight  f^x^t 
M(c),<r*(c))  on  the  observation  xf  so  that  its  contribution  to  the  estimates  of  n  and 
<jz  will  be  small .  This  particular  x1  will  almost  always  require  further  study  or 
attention  but  In  some  cases,  as  In  this  paper,  these  observations  need  to  be  removed 
from  the  original  data  set  as  well  as  Investigated.  The  action  of  the  screen  Is  not 
limited  to  a  single  observation,  we  have  found  these  methods  to  be  useful  In  practice 
even  when  as  many  as  40*  of  the  original  sample  were  highlighted  as  not  consistent 
with  a  single  Gaussian  population. 

The  estimating  equation  (2.5)  may  be  regarded  as  a  differential  equation  whose 
solution  may  be  shown  to  be  the  objective  function,  the  generalized  likelihood , 


•c 


1  £  ffc(Xt:  n,9*) 

®  »=!'■  Q*(M,C*,C) 


C  *  0, 


(2.7) 


where  a  =  c/(  l+c ) .  The  objective  function  fc  is  a  generalization  of  the  log 
likelihood;  Indeed  the  limit  of  lc  as  c  tends  to  zero  reduces  to 


n 

«o  =  £  log  f(*i  !  (2.8) 

1=1 

the  usual  log  likelihood. 

The  expression  (2.7)  is  easily  extended  to  cover  a  Gaussian  error  structure 


combined  with  functional  structure.  Accordingly  we  shall  develop  an  objective  function 
for  the  two-way  analysis  at  varlanca  layout  with  Intaractlon  and  empty  calls.  Tha 
variation  at  tha  uaar  spadflad  c  parameter  corresponds  to  variation  in  tha  way  tha 
Information  is  extracted  froa  tha  saapla  to  arrive  at  model  specification. 

Tha  family  at  estimators  or  modal  summarization*  which  result  froa  variation  at  tha 
user -specified  coefficient  c  should  not  change  such  If  tha  underlying  tentative  modal 
and  the  data  are  internally  consistent.  indeed,  extensive  simulation  trials  completely 
corroborate  this  statement i  on  tha  other  hand,  if  model  and  data  are  not  consistent, 
the  estimates  can  vary  dramatically .  This  is  particularly  true  if  tha  data  contalm 
outliers  vls-a-vls  the  error  distribution  assumption.  Because  all  at  our  analyses 
center  an  the  processing  at  sample  information  to  arrive  at  a  summarization  of  a 
tentative  model ,  we  will  call  the  procedure  which  enables  this  self-  or  model-  critical. 
Models  are  evolved  when  this  process  of  model -criticism  (Box,  1979 1  Daniel  1978  j 
Paulson  and  Nlcklln,  1983;  Paulson  and  oelehanty,  1983)  shows  that  the  data  and  the 
model,  both  error  and  function  components,  are  not  internally  or  mutually  consistent. 

As  we  shall  see,  an  appropriate  model  for  the  hemotology  testing  program  Is 

yijk  =  l°g(x13*)  =  M  +  a,  +  0J  +  y,j  +  u1jK,  (2.9) 

for  1  =  1,2,..,  I,  J  =  1,2,...,  J ,  *  =  1,2,...,  n1J(  and  where  u ,  j  k  are 
independent  normal  (Gaussian)  with  mean  zero  and  variance  a* ,  m  short  u,Jk  are 
N(0 ,cr* ) .  The  represent  the  prothrombin  time  for  the  kth  laboratory  using 

method  1  and  thromboplastin  j  In  our  context.  The  generalized  likelihood  for  the 
model  (2.9)  is 


5 


where 


f(u,jk)  »  exp{-|  (*tjk  -  M  -  «i  -  0j  -  yij)2/^*} 

and  Q  is  as  before.  on  differentiating  fc  with  respect  to  n,  a,  ,  0j  ,  y,  j ,  and  er* 
and  setting  the  resulting  expressions  to  zero  we  obtain  the  system  of  implicit  equations 

CEE  (ytjk  -  4  -  at,  -  /3j  -  y,  j  )  v1jk(c)  =  0  (2.1ia) 

t  J  k 

E  E  (yi3k  -  M  -  y<  3  )  Vijk(c>  =  0.  i=l,2,...,I,  (2.11.b) 

j  ^ 

E  E  (yijk  -  4  -  -  y(j)  v1Jk(c)  =  0,  J=l,2 . J,  (2.11.C) 

E  (y(jk  -  m  -  <*i  -  flj  -  y<j)  vijk(c)  =  o,  i=it2,. ...i  ( 2 . n .d > 

h  J=l,2 . J, 

<1+C)  EEC  ( y t j k  -  4  *  a,  -  0]  -  yij)*  v1Jk(c) 

a2  =  - LL± -  ,  (2.ll.e) 

E  E  E  v, jk(c) 

1  J  k  1 


and 


v1Jk(c)  =  expj-|  (y1Jk  •  m  -  a,  -  p}  -  y,j)2/cT2J  .  (2.ll.f) 

The  system  (2.11)  is  not  of  full  rank  for  cao.  In  order  to  obtain  a  full  remit  system 
we  impose  the  constraints  •  and  these  are  the  most  natural  • 


E  a,v,  .  .(c )  =  0,  C  0,v...(c)  =  0, 
i  J  1  1 

E  yijv,j.(c)  =  0,  J=1.2 . J, 

E  ytjv, . .(c)  =  o,  i=i,2, ... ,r, 

) 

and  where  a  dot  indicates  summation  over  a  subscript,  eg., 


( 2 .12 .a) 

( 2 . 12 .b) 

(2.12.C) 


This  lyatm  of  equations  la  solved  by  recursion.  Prog ra»s  are  available  on  request 


froa  the  authors. 

Equations  (2. 9) -(2. 12)  determine  the  generalised  likelihood  or  model -critical 
analysis  for  a  two-way  layout  with  (n,  ^  =  o  for  some  1  and  j)  or  without  (n^  >  0) 
empty  cells.  When  cells  of  the  array  are  empty,  the  same  equations  are  applicable 
but  with  some  minor  modifications  to  be  discussed  In  section  4  in  the  context  at  the 
analysis  of  the  prothrombin  time  data. 

Typical  values  of  c  to  be  used  In  the  model -critical  procedure  Involving  a  two-way 
layout  with  a  normal  error  distribution  are  0  *  c  *  0.50.  As  c  varies  from  c  =  o  to 
c  =  .1,  .2,  .3,  for  example,  the  parameter  estimates  surface  9(c)  =  (m(c),  a,(c), 

0j(c),  yij(c),  <r*(c),  1  =  1,2,...,  I,  J  =  1,2 . J),  say,  will  all  vary  as  a 

function  of  c.  We  need  to  be  able  to  objectively  assess  whether  the  response  surface 
varies  statistically  significantly  as  a  function  of  c .  The  next  section  provides 
procedures  for  making  this  assessment. 


3.  Tests  of  Pit 

Plrst  we  consider  the  case  when  xx,x2,..,xn  is  tentatively  taken  to  be 
independent  N  (  m  ,  <J"* )  •  The  process  of  varying  the  way  the  Information  In  the  data  la 
summarised  in  the  parameters  jx  and  cr2  through  the  generalized  likelihood  will  lead  to 
families  of  summarization*  u(c)  and  o2(c).  Por  two  different  values  of  c,  c  and  c', 
say,  the  values  of  m(c)  and  m(c'),  ai(c)  and  can  be  different.  These 

differences  can  be  used  to  develop  a  statislcal  test  of  fit  for  the  appropriateness  at 
the  model.  Given  the  estimates  m(c)  and  <r2(c) ,  an  estimate  of  the  model  density  is 

f(  X:  m(C),<T2(C))  =  (2rrcr2(c  )  )_l*  exp<  - 1|(  x-M(  C  )  )2<t2(  c  )  ) .  (3.1) 


7 


This  estimate  of  the  model  density  captures  ths  sample  Information  provided  the 


tentative  Gaussian  model  is  correct.  A  test  of  appropriateness  of  the  Gaussian  model 
for  the  data  xA  ,x2 . xn  can  be  based  on 

D(  1 1 2  >c )  =  n  f°  (f(x«  A.  A*)-f(xi  ti(c),a*(c))  log  — — -’■<T. ) -  dx  (3.2) 

f(x«  A<c),a*(c)) 

for  sosw  c*0 ,  where  D(lt2tc)  represents  an  information  divergence  (JC all  bade ,  1959, 
Chapters  l  and  2)  based  on  the  assumed  model  and  the  data  as  summarized  in  the 
estimators  m(c)  and  cr2(c),  c*0.  We  have  extensively  investigated  this  statistic  as  a 
test  of  fit  for  a  variety  of  values  of  c  and  found  that  it  does  Indeed  make  for  a  good 
test  of  Gausslanity.  However,  the  percentage  points  of  this  statistic  are  appropriate 
for  the  case  of  testing  that  xi,zI,...,  xn  are  Independent  Gaussian  but 

would  not  be  appropriate  for  testing  a  model  with  combined  Gaussian  error  and 
functional  structure  since  the  required  percentage  points  depend  on  the  specifics  of 
the  functional  model  structure.  Thus  0  could  not  be  used  to  test  the  appropriateness 
of  (2.9)  for  prothrombin  times. 

However,  another  Information  divergence  which  explicitly  depends  only  on 
estimated  variances  <j*  ( c ) ,  and  which  does  not  depend  on  the  specifics  of  the 
functional  structure  of  the  model  is 

J(  1 : 2 :  c  )  =  n  |  (f(xi  O.a2)  -  f(x:  0,a2(c))  log  — -  dx  (3.3) 

J  *®  f(x:  0,ct2(c)) 


n  f  <t*  +  <7*(c ) 

2  !■<>*(  c)  a 


2 


The  statistic  J  represents  the  divergence  between  an  estimate  of  a  Guasslan  density 
based  on  the  maximum  Likelihood  estimate  of  the  variance  and  an  estimate  of  the 
Gaussian  density  based  on  the  generalized  likelihood  estimate  of  variance.  m  this 
case  any  common  mean  substituted  for  zero  in  (3.3)  will  yield  the  same  result.  This 


doe*  not  Imply  that  (3.3)  la  Independent  of  the  estimate*  of  the  naan  a a  may  be  aaan 
froa  system  (2.6). 

The  statistic  (3.3)  can  ba  viewed  aa  an  analogue  of  the  shapiro-mik  w  atatlaUc 
for  testing  for  Gmsslanlty  (Shapiro  and  Wllk,  1965).  Tha  ratlnala  behind 

«  *  <3*> 


where  x1,x,,...f  xn  la  putatlvaly  N(m.<7*)  and  tha  aj  are  tabulated  conatanta 
(Shapiro,  i960),  la  that  tha  numerator  and  denominator  of  (3.4)  are  both  estimate# 
of  a  constant  multiple  of  <r*.  If  one  of  tha  estimator*  la  markedly  different  from  tha 
other,  small  values  of  w  will  result  and  evidence  against  the  hypothesis  of  Giuaalanlty 
will  ba  strong. 

The  rationale  behind  tha  nonnegative  statistic  J(lt2ic)  of  (3.3)  la  similar  and  la 
as  follows.  if  xt,xt,...,  xn  are  Independent  W(m,c*),  then,  apart  froa  sampling 
error,  both  <ra  and  <ra(c)  are  estimator*  for  a*.  Large  values  of  J(li2»c)  will 
provide  evidence  against  the  hypothesis  of  Gausslanlty.  The  statistic  J(li2ic)  Is 
particularly  sensitive  to  outlier -like  departure*  froa  Gausslanlty  since  the  Influence 
curves  (see  Barnett  and  Lewis,  1978,  pp  136-142)  at  observation  xj  for  the 
estimators  of  and  cr*  at  the  univariate  Gaussian  density  are  proportional  to 

>{(!«)  »°«  .  «°«01 


ae 


I 


.  c  >  o. 


for  0  =  n  and  <x*  respectively.-  Therefore,  In  the  process  of  the  generalised 
likelihood's  adaptation  to  the  best  summarisation  of  the  data  consistent  with  the 
tentative  model  of  Gausslanlty,  the  Influence  of  an  outlying  observation  or  groups  of 
outlying  observations  will  be  ultimately  downweighted  by  fc( xji  n(c)  ,<ra(c) ) .  For  xj 
far  removed  froa  ji(c)  in  comparison  with  the  scale  ff(c),  f°( xjt  y(c),<r2(c))  will  be 
nearly  sero. 
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Table  1  provide*  the  upper  percentage  point*  for  the  statistic  J(it2ic)  for  c  = 
-.2, .2, .3, .4, .5  and  for  n  =  10,20,24,30,40,60,120,460.  These  percentage 
points  have  been  developed  as  follows.  First,  10,000  independent  realizations  of 
J(  1 1 2 1  c )  were  slaulated  for  each  value  of  c  and  n.  Nest  these  realisations  were  put 
In  ascending  order  and  appropriate  estimates  of  the  percentage  points  were  tabulated. 
Finally,  cubic  spline  functions  were  fit  to  the  resulting  surfaces  In  sample  else  n, 
coefficient  c,  and  size  of  test  a.  completely  Independent  simulation* ,  1 . e . , 

Independent  programmers  and  programs ,  were  used  to  check  the  simulations  which 
produced  Table  l. 

In  order  to  determine  the  dependence  of  the  percentage  points  J  of  Table  l  on 
the  specifics  of  a  functional  model  sturcture ,  the  percentage  points  of  J  were 
simulated  under  a  variety  of  regression  structures,  linear  model  structures,  and 
nonlinear  functional  model  structures,  all  under  the  assumption  of  an  additive  Gaussian 
error  structure.  m  all  cases  the  percentage  points  of  the  statistic  J  were  the  same 
as  those  computed  under  the  model  assumption  that  a  Gaussian  error  structure  alone 
describes  the  data,  l.e.,  xlfx2,...,  xn  jure  independent  N(ji,  cr*),  apart  from 
sampling  error  and  a  correction  which  accounts  for  the  number  of  parameters 
estimated .  This  Independence  of  J  of  the  functional  structure  Is  suggested  by  the 
similar  property  of  the  3haplro- Milk  statistic  but  we  have  not  succeeded  In  developing 
an  analytical  proof. 

Consider  the  modeling  framework  where  it  Is  postulated  the  responses  of  interest 
follow  the  tentative  functional  and  error  representation 

Y\  ~  ^ il , ^il ,  •  •  •  ,  *mi  1  »®i » •  •  • »  ^  ,  1  ~  1,2 .  n, 

where  h  Is  some  specified  function  which  may  simply  be  constant  n ,  the  additive  errors 
ej  are  Independently  and  Identically  N(0  ,ct2  ) ,  the  xjj  are  measurements  on  variables 
which  may  Influence  the  y\,  and  ,  dq  are  parameters  to  be  estimated  from 


the  data.  The  tentative  model  will  be  fit  by  generalised  likelihood  for  c*0  and  km 
OO,  say  c*.3,  with  resulting  estimates  ®i(c)  and  <x*(c).  The  statlatlc  J(l»2ic)  la 
thla  aattint  computed  aa 


J(  1 1  2  rC  ) 


j*(c) 
a * 


2J 


and  raforrad  to  Tabla  l.  if  J( 1 i 2 ic )  exceeds  tha  tabulated  critical  valua  at  laval 
a,  than  it  la  poaalbla  to  improve  tha  tantatlva  nodal  in  tha  aanaa  that  some  data  may 
ba  tnconalatant  with  tha  nodal,  tha  functional  atructura  of  tha  nodal  nay  naad  to  ba 
evolved,  tha  arror  atructura  at  tha  nodal  nay  naad  to  ba  avolvad,  or  both  arrar 
atructura  and  functional  atructura  ahould  ba  axamlnad  and  avolvad ,  ate .  in  aona 
caaaa  conaldarabla  atudy  may  ba  required  bafora  tha  sourca  of  a  statistically 

significant  J( 1 1 2 ic )  is  found  and  a  choica  for  avolutlon  of  tha  modal  is  nada, 

although  In  nany  caaaa,  such  as  hara  for  prothronbln  tlnas,  tha  sourca  at  a 

statistically  significant  J( 1 i 2 ic)  may  ba  raadlly  found.  Standard  graphical  and 
diagnostic  procadura  ahould  ba  usad  in  conjunction  with  tha  ganarallaad  likelihood,  and 
tha  ganarallsad  llkallhood  residuals  and  tha  J  teat,  indeed,  we  regard  all  aspects  of 
tha  ganarallsad  llkallhood  as  complementary  to  tha  standard  procedures  of  data 
analysis  and  statistics .  We  note  In  passing  that  tha  statistic  J  of  (3.3)  Is  readily 
extended  to  tha  p- variate  case. 

even  though  tha  statistic  J( 1 1 2 ic)  has  been  developed  to  aid  in  model  assessment 
and  avolutlon.  It  provides  for  a  good  pure  test  for  normality.  Table  2  presents 
powers  of  the  J,  skewness  bA,  kurtoels  bs,  the  Anderson  -  Darling ,  and  the 

Shapiro -Wllk  tests  for  normality  under  the  alternatives  of  a  heavily  right -skewed 
lognormal  distribution,  t  distributions  on  n  degrees  of  freedom  (designated  T(m ) ) , 
chi -squared  distributions  m  a  degrees  of  freedom  (designated  x*(b>).  and  four 


mixture  alternatives,  The  mixture  alternatives  are  as  follows!  Mxi  is  75* 


H(0,1)  and  25*  N(  2 , 1 )  I  Mx2  la  SO*  H(0,1)  and  SO*  N(2,l)»  Mx3  la  SO*  H(0,1> 
and  50*  N(0,4)|  Mx4  la  50*  N(0,l)  and  so*  N(i,i).  Mx4  la  a  particularly  sever* 
test  of  any  test  for  normality  since  the  existence  of  the  mixture  can  be  very  difficult 
to  detect,  we  have  provided  tabulations  In  Table  l  and  Table  2  of  values  c  =  -0.2 
and  -0.1  because  the  use  at  negative  c  In  the  generalised  likelihood  has  been  found 
to  be  useful  In  several  applications  Involving  In -lying  contamination  of  data  and 
multiple  clusters  at  data.  Tabulations  at  powers  In  Table  2  Indicates  that  J(  1 1 2 ic) 
provides  for  a  good  test  at  normality  for  a  wide  range  of  c.  The  Importance  of 
J(  1 1 2 ic ) ,  however,  Is  due  to  its  linking  estimation  and  asssosmontB  of  fit. 

Example.  m  a  study  concerning  tests  for  outliers,  Quesenberry  and  David  (1961) 
provide  the  sixteen  observations  .  32 , .  35 , .  37 , .  38 , .  39 , .  44 , . 45 , . 46 , . 47 , . 48 , .  52 , .  S3 , 
.57,.  74,.  74,  1.09  In  Illustration  of  a  studentlxed  range  test.  This  studentised  range 
test  finds  the  observation  1.09  to  be  too  large,  but  Just  barely,  to  be  consistent  with 
the  xlfX|,...,  xie  being  independent  and  Identically  Gaussian,  we  find  a*  =  al(0) 
=  0.0357,  o"*(  .  3 )  =  0.0134  and  J(  1 1 2 ic)  =  8.32.  Comparison  of  this  value  of  J 
with  the  critical  values  at  Table  l  shows  that  this  data  Is  dramatically  non -normal,  a 
finding  visually  corroborated  by  a  normal  probability  plot.  The  studentised  range  test 
experiences  difficulty  In  rejecting  the  observation  l .  09  as  an  outlier  because  the 
outlying  nature  of  this  data  point  is  being  Inter -mixed  with  the  otherwise  dramatic 
non -normality  of  the  data  and  the  value  1.09  Is  also  dramatically  Influencing  the 
estimated  variance  of  the  sample. 

The  J( 1 >2 ic)  test  statistic  is  useful  In  evaluating  the  statistical  status  of  a 
tentative  modeling  structure  and  in  determining  whether  a  tentative  model  should  be 
evolved.  Furthermore,  the  use  of  diagnostic  tools  stemming  from  the  generalised 
likelihood  procedure  and  other  diagnostic  tools  will  usually  be  helpful  in  determining 
the  direction  of  the  evolution. 


Tabl*  i.  Psrcsntat*  Points  at  ths  Ttat  Statistic  J(ii2tc)  for  -0.2  *  c  *  o.S 


Saapls  Sirs  0.75 

10  0.38 

20  0.46 

24  0.48 

C*0 .5  30  0.49 

40  0.S1 

60  0.53 

120  0.54 

480  0.55 

IO  0.24 

20  0.30 

24  0.31 

CaO.4  30  0.32 

40  0.34 

60  0.36 

120  0.37 

480  0.37 

10  0.13 

20  0.17 

24  0.18 

C=0 . 3  30  0.19 

40  0.20 

60  0.21 

120  0.22 

480  0.22 

10  0.059 

20  0.078 

24  0.082 

C*0.2  30  0.087 

40  0.092 

60  0.099 

120  0.11 

480  0.11 

10  0.015 

20  0.020 

24  0.021 

C^J.l  30  0.023 

40  0.024 

60  0.027 

120  0.029 

480  0.030 


SlE#  Of  TSSt 


0.80 

0.85 

0.90 

0.45 

0.60 

1.61 

0.56 

0.  70 

1.11 

0.58 

0.74 

1.10 

0.60 

0.75 

1.07 

0.63 

0.80 

1.06 

0.65 

0.80 

1.05 

0.67 

0.84 

1.10 

0.69 

0.87 

1.10 

0.28 

0.34 

0.61 

0.  36 

0.45 

0.64 

0.38 

0.48 

0.66 

0.40 

0.50 

0.68 

0.42 

0.53 

0.70 

0.44 

0.55 

0.72 

0.46 

0.57 

0.75 

0.47 

0.60 

0.85 

0.15 

0.18 

0.23 

0.21 

0.28 

0.34 

0.22 

0.27 

0.36 

0.23 

0.28 

0.38 

0.24 

0.30 

0.42 

0.26 

0.32 

0.44 

0.28 

0.34 

0.45 

0.28 

0.35 

0.46 

0.068 

0.078 

0.095 

0.092 

0.11 

0.14 

0.097 

0.12 

0.15 

0.10 

0.13 

0.16 

0.11 

0.14 

0.18 

0.12 

0.15 

0.20 

0.13 

0.16 

0.21 

0.14 

0.17 

0.23 

0.017 

0.020 

0.023 

0.023 

0.028 

0.034 

0.025 

0.030 

0.037 

0.02  7 

0.032 

0.040 

0.029 

0.035 

0.045 

0.032 

0.039 

0.050 

0.036 

0.044 

0.057 

0.039 

0.047 

0.060 

0.95 

0.975 

0.99 

6.40 

15.7 

40.5 

2.70 

5.32 

10.8 

2.34 

4.40 

8.37 

2.16 

3.91 

6.95 

1.95 

3.31 

5.60 

1.68 

2.78 

4.50 

1.66 

2.33 

3.64 

1.65 

2.10 

3.11 

2.45 

6.92 

19.7 

1.60 

3.02 

6.33 

1.45 

2.64 

5.07 

1.35 

2.41 

4.40 

1.30 

2.20 

3.73 

1.12 

1.85 

3.15 

1.11 

1.65 

2.51 

1.09 

1.51 

2.10 

0.74 

2.11 

8.16 

0.77 

1.46 

3.19 

0.  77 

1.41 

2.87 

0.  77 

1.39 

2.47 

0.74 

1.31 

2.15 

0.69 

1.10 

1.82 

0.67 

0.96 

1.49 

0.67 

0.88 

1.26 

0.18 

0.48 

1.33 

0.27 

0.54 

1.29 

0.29 

0.56 

1.23 

0.31 

0.57 

1.12 

0.31 

0.58 

1.02 

0.32 

0.53 

0.92 

0.32 

0.47 

0.  75 

0.33 

0.46 

0.69 

0.031 

0.052 

0.11 

0.051 

0.095 

0.23 

0.056 

0.11 

0.25 

0.066 

0.12 

0.26 

0.072 

0.13 

0.25 

0.074 

0.13 

0.23 

0.085 

0.12 

0.21 

0.089 

0.12 

0.20 

it- 


Table  i.  percentage  Points  of  the  Teet  statistic  J( 1 1 2  tc)  for 
-0.2  a  c  a  0.5  (cont'd) 


Staple  Sice 

0.75 

0.80 

0.85 

0.90 

0.95 

0.975 

0.99 

10 

0.017 

0.019 

0.022 

0.025 

0.029 

0.034 

0.038 

20 

0.022 

0.025 

0.029 

0.035 

0.044 

0.055 

0.093 

24 

0.023 

0.027 

0.031 

0.038 

0.049 

0.063 

0.11 

ca-0.1  30 

0.025 

0.030 

0.035 

0.042 

0.054 

0.071 

0.13 

40 

0.027 

0.032 

0.038 

0.047 

0.063 

0.091 

0.17 

60 

0.030 

0.036 

0.043 

0.054 

0.075 

0.11 

0.20 

120 

0.034 

0.041 

0.052 

0.065 

0.094 

0.14 

0.24 

480 

0.038 

0.047 

0.060 

0.075 

0.11 

0.16 

0.28 

10 

0.077 

0.086 

0.097 

0.11 

0.13 

0.15 

0.16 

20 

0.097 

0.11 

0.13 

0.15 

0.18 

0.22 

0.28 

24 

0.10 

0.12 

0.14 

0.16 

0.20 

0.25 

0.33 

C=-0 . 2  30 

0.11 

0.13 

0.15 

0.18 

0.22 

0.27 

0.42 

40 

0.12 

0.14 

0.16 

0.20 

0.26 

0.33 

0.55 

60 

0.13 

0.16 

0.19 

0.23 

0.31 

0.40 

0.  70 

120 

0.15 

0.18 

0.22 

0.28 

0.40 

0.55 

0.97 

480 

0.17 

0.21 

0.26 

0.33 

0.49 

0.70 

1.23 

Table  2.  Comparative  Powers  of  the  J,  bA  and  bs,  the  Anderson  -  Darlln*  (A-D) 
and  Shaplro-wilk  ( 3 -* )  Statistics  for  Several  Alternatives 


(a)  Slse  a  «  0.1  and  n  =  20 


0.5 

LN 

0.84 

T(  9 ) 

0.22 

T<7) 

0.27 

T(5) 

0.34 

T(3) 

0.52 

T(l) 

0.93 

X*( 14 ) 

0.20 

X*(10) 

0.25 

X*(  6 ) 

0.32 

X*(  4 ) 

0.42 

Xa(  2 ) 

0.63 

X*<1) 

0.84 

Nxl 

0.10 

Mi2 

0.02 

N«3 

0.31 

Mx4 

0.09 

1* 

0.99 

T(  9 ) 

0.31 

T(7) 

0.40 

T(5) 

0.55 

T(3) 

0.80 

T(  1 ) 

1.0 

X*( 14 )  0.30 
X*(10)  0.37 
X* ( 6 )  0.50 

X* ( 4 )  0.65 

x*( 2 )  o.aa 

X*(l)  0.99 

Mx  1  0.10 

M«2  0.12 

Nx3  0.46 
Itx4  0.10 


0.4  0.3 

0.82  0.80 
0.22  0.21 
0.26  0.25 

0.34  0.34 

0.52  0.51 

0.92  0.92 

0.20  0.20 

0.25  0.25 

0.33  0.32 

0.42  0.40 

0.62  0.58 

0.82  0.79 

0.09  0.08 

0.03  0.05 

0.31  0.30 

0.08  0.08 


0.99  0.99 

0.31  0.30 

0.41  0.40 

0.56  0.55 

0.80  0.79 

1.0  1.0 

0.31  0.31 

0.38  0.37 

0.50  0.50 

0.65  0.64 

0.87  0.85 

0.98  0.98 

0.09  0.07 

0.13  0.14 

0.46  0.44 

0.10  0.10 


0.2 

0.1 

0.77 

0.72 

0.20 

0.19 

0.24 

0.22 

0.32 

0.29 

0.48 

0.46 

0.90 

0.89 

0.19 

0.18 

0.24 

0.23 

0.31 

0.28 

0.39 

0.36 

0.55 

0.51 

0.75 

0.69 

0.08 

0.08 

0.08 

0.12 

0.27 

0.23 

0.09 

0.10 

Slse 

a  a  0 

0.98 

0.97 

0.30 

0.29 

0.40 

0.38 

O 

ut 

0.52 

0.78 

0.  76 

1.0 

1.0 

0.30 

0.29 

0.36 

0.35 

0.48 

0.45 

0.61 

0.57 

0.83 

0 . 79 

0.97 

0 . 95 

0.07 

0.08 

0.15 

0.18 

0.41 

0.38 

0.10 

0.10 

-0.1  -0.2 


0.59 

0.52 

0.15 

0.12 

0.17 

0.15 

0.23 

0.19 

0.37 

0.32 

0.82 

0.78 

0.16 

0.15 

0.19 

0.16 

0.22 

0.20 

0.27 

0.24 

0 . 39 

0.34 

0.54 

0.45 

0.08 

0.09 

0.18 

0.20 

0.15 

0.11 

0.10 

0.10 

1  and  n 

a  50 

0.92 

0.87 

0.24 

0.22 

0.31 

0.28 

0.47 

0.41 

0. 71 

0.64 

1.0 

1.0 

0.25 

0.23 

0.29 

0.27 

0. 38 

0.33 

0.47 

0.42 

0.68 

0.61 

0 . 87 

0 . 81 

0.08 

0.09 

0.24 

0.27 

0.28 

0.24 

0.11 

0.11 

bl  b2 


1.0  0.94 

0.23  0.24 

0.29  0.32 

0.41  0.47 

0.58  0.72 

0.92  1.0 

0.59  0.25 

0.69  0.31 

0.86  0.40 

0.95  0.50 

0.99  0.72 

1.0  0.91 

0.22  0.08 
0.03  0.25 

0.26  0.31 

0.08  0.11 


A-D  S-W 

0.94  0.96 

0.17  0.15 

0.19  0.19 

0.24  0.24 

0.41  0.42 

0.90  0.88 

0.24  0.28 

0.31  0.34 

0.43  0.48 

0.58  0.64 

0.86  0.91 

0.99  1.00 

0.16  0.15 

0.10  0.09 

0.22  0.21 
0.10  0.10 


1.0  1.0 
0.19  0.16 

0.26  0.23 

0.40  0.35 

0.69  0.61 

1.0  1.0 
0.50  0.56 

0.59  0.69 

0.82  0.89 

0.94  0.98 

1.0  1.0 
1.0  1.0 
0.25  0.25 

0.14  0.19 

0.30  0.21 

0.11  0.11 
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T*bl«  2.  Comparative  Powara  at  tba  J,  bx  and  b, ,  t ha  Andaraon - Darlln*  ( A-D) 
and  Shapiro -Wllk  (3-W)  StatlaUca  for  savaral  Altamatlvaa  (oont'd) 

(c)  Six*  a  *  0.1  and  n  *  100 


0.5 

LN 

1.00 

T(9) 

0.41 

T(7) 

0.56 

T(  5  ) 

0.76 

T(3) 

0.95 

T(l) 

1.00 

X*(14) 

0.36 

X*( 10 ) 

0 . 49 

X*(  6  ) 

0.67 

X*<4) 

0.84 

X*(  2 ) 

0.98 

X*(l) 

1.00 

Nxl 

0.10 

Mx2 

0.28 

Mx3 

0.66 

Mx4 

0.09 

0.4 

0.3 

1.00 

1.00 

0.42 

0.42 

0.57 

0.57 

0.  77 

0.77 

0.95 

0.95 

1.00 

1.00 

0.38 

0.39 

0.49 

0.50 

0.68 

0.68 

0.84 

0.83 

0.98 

0.98 

1.00 

1.00 

0.09 

0.08 

0.29 

0.29 

0.65 

0.63 

0.09 

0.09 

0.2 

0.1 

1.00 

1.00 

0.43 

0.42 

0.57 

0.55 

0.  76 

0.74 

0.95 

0.94 

1.00 

1.00 

0.40 

0.38 

0.50 

0.49 

0.65 

0.62 

0.82 

0.79 

0.97 

0.95 

1.00 

1.00 

0.08 

0.08 

0.30 

0.31 

0.60 

0.57 

0.10 

0.10 

0.1 

-0.2 

0.99 

0.98 

0.38 

0.35 

0.51 

0.46 

0.70 

0.65 

0.92 

0.88 

1.00 

1.00 

0.34 

0.31 

0.43 

0.38 

0.55 

0.49 

0.71 

0.64 

0.91 

0.86 

0.99 

0.97 

0.08 

0.10 

0.36 

0.39 

0.48 

0.42 

0.11 

0.12 

bl 

b2 

1.00 

1.00 

0.28 

0.37 

0.35 

0.50 

0.51 

0.70 

0.71 

0.93 

0.96 

1.00 

0.86 

0.34 

0.94 

0.44 

0.99 

0.57 

1.00 

0.73 

1.00 

0.93 

1.00 

0.99 

0.36 

0.09 

0.02 

0.39 

0.30 

0.49 

0.08 

0.11 

A-O 

S-W 

1.00 

1.00 

0.29 

0.43 

0.36 

0.54 

0.61 

0.74 

0.89 

0.94 

1.00 

1.00 

0.75 

0.85 

0.86 

0.94 

0.98 

1.00 

1.00 

1.00 

1.00 

1.00 

1.00 

1.00 

0.40 

0.36 

0.22 

0.15 

0.47 

0.56 

0.10 

0.11 
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4.  Results  of  t he  Prothrombin  Teeting  Prog  r as 


Approximately  400  laboratories  participate  In  the  New  York  Department  of  Health 
prothrombin  time  proficiency  testing  program .  This  paper  will  focus  on  the 

approximately  320  of  these  400  laboratories  which  use  automated  testing  methods .  The 
data  analysed  in  this  study  were  reported  for  12  test  specimens,  3  in  each  of  4 
specimen  mall  outs  >  January,  July,  and  October  of  1982  and  July  1983.  The 
laboratories  which  utilised  automated  testing  methods  used  various  combinations  of  eight 
analysis  methods  and  nine  thromboplastins.  For  example,  for  the  July  1983  mall  out, 
318  automated  laboratories  used  47  combinations  of  methods  and  thromboplastins  as 
indicated  in  Table  3.  Twenty -five  combinations  of  method  and  thromboplastin  were  not 
utilised  by  any  laboratory. 

Table  3  suggests  that  an  appropriate  model  for  the  analysis  of  the  clotting  times 
Is  a  two-way  analysis  of  variance  layout  with  interaction  and  with  an  error  distribution 
to  be  determined.  The  development  of  a  tentative  error  model  for  the  responses  from 
the  participating  laboratories  was  facilitated  by  a  special  identification  study.  tn  this 
special  study,  13  reference  laboratories  (those  with  a  reputation  for  excellence)  were 
sent  three  specimens .  Each  reference  laboratory  was  instructed  to  report  triplicate 
measurements  on  the  specimens  obtained  with  each  of  the  9  types  of  thromboplastins 
supplied  in  the  mall  out  kits.  Analysis  of  the  special  study  data  showed  that  a  two  way 
layout  provided  a  reasonable  tentative  model  and  that  the  logarithm  of  prothrombin 
times  provided  normal  and  homoekedastic  residuals. 

Therefore  the  tentative  model  assumed  is  that  the  participating  laboratories  will 
produce  prothrombin  times  x1]H  which  follow  the  model  (2.9)  where  x1]k  Is  the 
prothrombin  time  for  method  1,  thromboplastin  J,  1=1,2,...,  8,  J=l,2,...,  9,  and 
k=i  ,2 ....  ,  n,j  a  0.  The  quantity  is  the  overall  c  grand  mean,  a,  is  the  effect 
due  to  method  i,  is  the  effect 


Table  3 


Method  Cod— i  1,  Autofl  ( Date )  ;  2,  Clot**  ( Hyland) ;  3,  Elbroneter  (BloQu**t)i  4, 
Coag-A-Mate  150  or  Dual  Beam  (General  Diagnostic )  i  5,  Coag-A-Mate  2001  or  X2 
(General  Diagnostics)  ;  6,  coagulation  Profiler  (Bio  Data) ;  7,  Coagullxer  ( Sherwood ), 
8,  Electra  (Medical  Laboratory  Autonation). 


Thronboplatln  Codes i  A,  Dade  Activated  Liquid  $  B,  Dade  C;  c,  Dade  Reagent;  D, 
Hyland  Dried;  E.  Hyland  Liquid;  F,  Ortho;  G,  Slmplastln  (General  Diagnostics);  H, 
simplastln  A;  I,  Slnplastln  Automated. 
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due  to  the  combination-  of  method  i  and  thtromboplastln  J .  The  u1  ]k  ere  tenatlvely 
normal  with  Man  zero  and  standard  deviation  <r. 

The  generalized  likelihood  for  model  (2.9)  with  all  n,  j  >  0  la  given  In  (2.10). 
Table  3,  however ,  ahowa  that  a  nuater  at  the  ntJ  are  o,  that  la  cell*  (l.A) ,  (i,C) , 

( 1 , F) , . . .  are  empty  eo  that  tha  generalized  likelihood  la  not  directly  applicable, 
wa  overcome  thla  difficulty  for  c>0  aa  follow* .  To  call  ( l , A)  wa  assign  a  large 
artificial  v&lua  jc ,  K>0|  to  call  (l,c)  wa  aaalcn  tha  artificial  v&lua  - jc *  to  call  (1,K) 
wa  assign  tha  v&lua  +2K  t  to  call  ( 1 ,  H)  wa  assign  tha  value  -2JC  j . . . .  Tha  procaaa 
continue*  until  all  empty  cell*  have  been  aaalgnad  an  artificial  value .  A  aul table 
magnitude  for  K  la  determined  by  the  location  and  spread  at  the  basis  data.  tn  our 
case  all  the  logarithms  at  prothrombin  times  are  at  the  order  3  with  a  standard 
davlatlon  of  order  .1  so  that  a  value  of  K*10  would  suffice .  Tha  strategy  at 
assigning  Isolated  artificial  values  +K,  -JC ,  *2K,  ...  is  required  In  order  to  preclude 
tha  formation  of  a  cluster  of  artificial  outlier*.  m  the  marl  ml  ration  of  (2.10) 
modified  by  inclusion  of  artificial  outlier*,  the  Influence  these  artificial  outlier*  exert 
on  the  estimate*  of  u,  a , ,  3j  ,  y(  j  and  a *■  will  be  at  most  of  order 


a  quantity  which  la  by  construction  virtually  aero.  This  procedure  for  missing  value* 
require*  that  every  row  and  every  column  contain  at  least  one  non-empty  cell. 

The  aodal -critical  or  aelf -critical  two-way  analysis  of  variance  procedure  teas 
applied  to  tha  twelve  mailouts  mentioned  above .  We  provide  only  representative 
Timmirlir*  of  these  analyse*.  Table  4  provide*  tha  results  for  grand  meana  u  and 
standard  deviations  a  at  the  analyse*  for  c  =  .1,  .2,  .25,  and  3  and  for  the 
missing  value  analysis  of  variance  (ANOVA).  m  every  case ,  the  grand  means  remain 
virtually  the  sir*  across  the  usual  least  square*  anova  and  generalised  livelihood 


analysis  with  c  =  . l,  .2,  .25,  and  .3.  Also  in  every  caee,  the  estimate  of  the 

standard  deviation  decreases  dramatically  with  an  increase  In  e  to  o.i  and  decreases 
Utile  thereafter. 

For  c  *  .3  and  sample  number  1,  for  example,  we  compute 


246  I” 

f-049]*  + 

f-o7ir  -2i 

2  [ 

l .  071 J 

1.049J  zj 

=  70.8, 

which  is  strongly  statistically  significant  according  to  Table  1  and  therefore  Indicates 
that  the  model  and  data  are  not  consistent  as  they  stand.  The  multiplier  246  is 

n-( I-KJ-l ) -(I-l )( I-l )  where  n  *  318,  the  number  of  participating  laboratories,  I  = 

8,  the  number  of  methods,  and  J  =  9,  the  number  of  thromboplastins .  The  decrease 

in  estimated  standard  deviation  Is  due  to  the  removal  of  the  influence  a t  the  out  at 

control  laboratories,  whose  prothrombin  times  are  not  consistent  with  the  two-way 
layout  with  Interaction  and  with  a  normal  error  distribution .  These  out  at  control 
laboratories  are  easily  determined  by  an  examination  at  the  generalized  likelihood 
residuals. 

Similar  results  concerning  standard  deviation  are  obtained  for  all  twelve 
mallouts.  This  behavior  Is  at  course  related  to  the  estimates  of  m,  and  a,  ,  /3}  ,  and 
ytj  for  (missing  value)  maximum  Likelihood  or  ( missing  value)  least  squares  and  for 
the  generalized  likelihood.  Sample  12,  the  last  sample  In  the  mallout  at  July  1983 
provides  a  typical  Illustration  of  the  behavior  of  the  estimates  of  the  parameters  a( 
and  a,  as  we  move  from  least  squares  to  generalized  likelihood  (c  =  0.25).  These 
results  are  presented  in  Table  5.  similar  results  obtain  for  the  estimates  of  the  y1}. 
These  variations  asm  use  their  true  Importance  when  the  estimates  are  all  combined  to 
obtain  the  predicted  cell  means .  Once  the  cell  means  and  the  estimated  standard 
deviation  are  available,  standardized  grades  for  each  laboratory  are  obtained.  A 


standardized  trade  la  the  number  o f  standard  deviations  by  which  a  particular 


laboratory  differs  from  the  predicted  cell  nean.  For  example,  if  a  Laboratory's 
prothroabln  time  results  in  a  standardized  grade  of  4-3,  this  Indicates  that  the 
laboratory's  reported  tlae  ms  3  estimated  a  units  above  the  cell  (i.e. ,  method  and 
throsboplastln  combination )  nean . 

The  selection  of  a  critical  value  beyond  which  a  laboratory's  performance  would 
be  considered  out  of  control,  required  the  responsible  decision  makers  to  trade  off 
two  types  of  error.  Dollar  costs  or  another  single  measurement  unit  for  the  relative 
weight  to  be  assigned  for  falsely  falling  a  satisfactory  lab  versus  falsely  passing  an 
unsatisfactory  lab  were  and  are  not  available.  Thus,  the  selection  of  a  critical  value 
was  based  on  the  Judgment  of  the  program  administrators  after  a  review  of  the 
implications  of  the  various  alternatives.  Program  administrators  settled  on  a  type  I 
error  level  of  2%.  Thus,  the  critical  Interval  was  set  at  +2 . 3  standard  units  about 
each  cell  mean.  This  reprsented  a  substantial  reduction  in  the  size  of  the  Interval 
froa  the  three  standard  units  that  had  been  used  with  the  prior  grading  system. 

The  practical  effect  of  the  application  of  self -critical  estimation  on  the  grading 
can  be  Illustrated  by  considering  the  differences  in  the  Laboratories  falling  outside  the 
critical  region.  In  Table  6  we  give  a  list  of  the  laboratories  falling  outside  the 
critical  region  using  least  squares  estimation  as  well  as  those  determined  from 
model  -critical  estimation  with  c=.25. 
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Table  4 


Estimates  at  generalised  likelihood  grand  mean*  and  etandard  deviation  a  at  model 

(2.9)  for  12  proficiency  study  mailoute. 


Sample 


Missing  Value 
ANOVA 


January  1982 

Grand 

mean 

2. 

1 

SO 

Grand 

mean 

2. 

2 

SO 

Grand 

mean 

3. 

3 

SO 

Model -Critical  Eatiaatee 
C^3.10  C*0 . 20  C^D.25  C=0.30 


44 

2. 

071 

.( 

87 

2. 

071 

.< 

05 

3. 

077 

. 

July  1982 


Grand  mean 
4  SD 


2.54 

.056 


2.54 

.056 


2.54 

.054 


2.54 

.054 


2.54 
.  053 


Grand  mean 
5  SD 


2. 76 
.071 


2.75 

.059 


2.75 

.058 


2.75 

.057 


2.75 

.057 


Grand  mean 
6  SO 


2.99 

.063 


2.99 

.059 


2.99 

.058 


2.99 

.057 


2.99 

.056 


October  1982 

Grand 

mean 

7 

SO 

Grand 

mean 

8 

SD 

Grand 

mean 

9 

SD 

July  1983 

Grand  mean 

2.49 

2 . 49 

2.49 

2.49 

2.49 

1 

10 

SD 

.06  3 

.051 

.050 

.049 

.048 

Grand  mean 

2.89 

2.89 

2 . 89 

2.89 

2.89 

11 

SO 

.06  7 

.056 

.054 

.054 

.053 

Grand  mean 

3  .  12 

3 . 12 

3.12 

3.12 

3.12 

! 

12 

SD 

.077 

.060 

.058 

.057 

.056 

22 

> 

1 

-  -  -  .*■'*.  *-  s'  S.', 

1  .  •’«  «’ _  s’ 

y-y  V\. 

» 

rihii  St  out  of  Control  Ladoraiorlee  i 
Coll  Nmm  and  Standardised  Grades 


Least 

squares 

Modal 

-Critical  ( c» . 25 ) 

a  • 

.055 

<?( 

.25)  »  .057 

Call 

standardised 

Call 

standardised 

call 

Maan 

cradaa 

Maan 

Gradea 

(3.P) 

3.171 

-5.57 

3.169 

-7.51 

2.73 

-3.67 

2.2S 

3.11 

-1.77 

-2.35 

(•,  B) 

3.051 

-7.  ** 

3.077 

-10.5* 

-1.9* 

-3.09 

2.23 

2.55 

2.  SO 

3.** 

(3.  A) 

3.0*7 

1.32 

3.1*9 

2 .  *0 

(3,  6) 

3.1*7 

2.03 

3.303 

2.92 

(5,  A) 

3.0** 

2 . 6S 

3.250 

*.52 

(1,  B) 

3.210 

1 . 31 

3.109 

-2.*? 
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note  that  only  5  of  318  laboratories  ara  Judged  aa  balnt  out  of  control  using  tha 
laaat  squares  estimates  (6.38  wsr*  expected)  while  with  tha  model -critical  procedure 
12  a f  318  observations  fail  in  tha  critical  ration.  This  la  tha  caaa  alnca  flrat,  undar 
laaat  aquaraa  estimation  with  alaalnf  value* ,  tha  outlying  observations  hava  tha 

graataat  lnfluanca  on  tha  paraaatar  sstimatss  and  aacond,  tha  itaratlva  procaaa 
involving  tha  aaalgnaant  of  axpactad  valuaa  to  empty  calla  Impart*  an  lnfluanca  to  tha 
*at1nata  of  tha  atandard  davlatlon.  Tha  aalf  -critical  procadura  Impart*  by  construction 
earo  lnfluanca  to  tha  aatlmata  of  tha  atandard  davlatlon.  whan  tha  out -of -control 
laboratorlaa  ara  raaovad  from  tha  data  baaa  and  tha  data  ra-analysad  in  accordanca 
with  tha  two-way  layout  with  normal  arror  and  lntaractlon  modal,  tha  paramatar 

aatlmata* ,  Including  <r( c ) ,  aura  virtually  unchangad  for  c  *  .  1 ,  .2,  . 25 ,  and  . 3 .  Tha 

ra-computad  valu*  of  J(li2ic)  la  no  longer  significant  for  any  value  of  c  In 

-0.2  *  c  a  O.S. 

The  analysis  and  results  described  for  mall  out  12  of  tha  New  York  state 
Department  of  Health  hematology  proficiency  tasting  program  ara  typical  of  those 
obtained  for  the  remaining  n  studies.  Dn  each  case  the  out -of -control  laboratories 
are  quickly  and  easily  Identified  with  resultant  high-quality  service  supplied  by  the 
laboratories . 


5.  conclusions 

The  grading  system  for  this  proficiency  testing  program  Is  aimed  at  detecting 
laboratories  whose  results  depart  fron  the  structural  and  error  modal  that  governs  the 
majority  of  tha  testing  laboratories.  Evan  after  the  form  of  tha  appropriate  arror 
modal  had  bean  Identified ,  parameter  estimators  robust  against  the  high  lnfluanca  of 
outlying  observations  are  essential.  Saif-  or  modal  critical  estimation  which  allow 
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for  the  simultaneous  robust  estimation  of  both  location  and  seal*  parameters  in  t ha 
presence  of  empty  calls  is  thus  wall  suitad  to  this  problem. 

this  approach  substantially  improved  tha  efficiency  at  tha  proficiency  testing 
grading  procedure  In  New  York  State  as  evidenced  by  tha  reduction  averaging  20*  In 
the  standard  deviation  estimates,  the  alaoet  routine  detection  at  out -of -control 
laboratories,  and  the  evolution  to  a  much  more  uniform  grading  procedure. 

The  nodal -critical  procedure  Incorporated  tha  tentative  nodal  In  making  an 
assasanent  at  the  internal  consistency  of  the  model  and  the  data  through  the 
parametric  modification  of  the  way  Information  concerning  a  tentative  nodal  la 
extracted  from  the  data,  m  tha  hemotology  testing  program  some  data  was  found  to 
be  Inconsistent  with  the  model  but  this  need  not  always  be  the  carnet  It  Is  often  the 
case  that  the  proposed  model  la  Inadequate  to  deal  with  the  data. 
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